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ABSTRACT/RESUME 

Public spending efficiency: institutional indicators in primary and secondary education 

This paper presents composite indicators of the institutional and policy characteristics of educational 
systems, collated from the questionnaire responses of 26 Member countries. These indicators provide an 
overview of the institutional framework in the primary and secondary education sector and are constructed so as 
to be used for the analysis of international differences in spending efficiency. The key features of the 
institutional setting in the non-tertiary education sector are grouped under three headings: i) the ability to 
prioritise and allocate resources efficiently (through decentralisation and mechanisms matching resources to 
specific needs); ii) the efficiency in managing spending at the local level (through outcome-focused policies and 
managerial autonomy), and Hi) the efficiency in service provision (through benchmarking and user choice). For 
each country, an intermediate indicator is computed for each of these six institutional properties. Composite 
indicators then combine the six intermediate indicators of spending efficiency into a single, aggregate measure. 
Results are presented and some of their implications are discussed. Overall, the characteristics of the 
institutional framework in the non-tertiary public education sector seem to be very favourable, compared to 
OECD average, in the United Kingdom, Australia, Norway, Denmark and the Netherlands, whereas results are 
less favourable for the Czech Republic, Greece, Luxembourg, Japan, Turkey, Hungary, Belgium (French 
speaking community), Switzerland and Austria. 

JEL Classification: Hll, H77, H83, 120, 128. 

Keywords: Public spending efficiency. Public education. Institutional indicators. Decentralisation, 

Benchmarking, User choice. Outcome-focused public policies. Managerial autonomy in the public sector. 



Efficacite de la depense publique : indicateurs institutionnels dans le secteur de I’education primaire 

et secondaire 

Ce document de travail presente sous forme d’indicateurs quantitatifs les reponses de 26 pays membres de 
TOCDE a un questionnaire portant sur Torganisation institutionnelle du secteur public de Teducation primaire 
et secondaire. Les indicateurs fournissent une vue d’ensemble des caracteristiques institutionnelles susceptibles 
de contribuer aux differences d’ efficacite de la depense publique entre les pays dans le secteur educatif. Les 
caracteristiques institutionnelles prises en compte sont regroupees autour de trois dimensions : i) la capacite a 
allouer efficacement les ressources consacrees a Teducation publique (decentralisation, prise en compte de 
besoins specifiques), ii) T efficacite de la gestion an niveau local (fixations d’objectifs, autonomic des ecoles), et 
Hi) T efficacite de la foumiture de service educatif an niveau local grace a des mecanismes de marche 
(evaluation des performances, role du choix de Tusager). Pour chaque pays, un indicateur intermediaire est 
calcule pour chacune de ces six caracteristiques institutionnelles. Un indicateur composite est alors construit qui 
foumit une mesure synthetique de la qualite des institutions du secteur public de Teducation an regard de leur 
capacite a renforcer Tefficacite de la depense publique. Les resultats montrent en particulier que les institutions 
educatives sont relativement favorables a Tefficacite de la depense publique an Royaume-Uni, en Australie, en 
Norvege, an Danemark et aux Pays-Bas ; et relativement defavorables en Republique Tcheque, en Grece, an 
Luxembourg, an Japon, en Turquie, en Hongrie, en Belgique (communaute francophone), en Suisse et en 
Autriche. 

Classification JEL : HI 1, H77, H83, 120, 128. 

Mots cles : Efficacite de la depense publique. Education nationale, Indicateurs institutionnels. Decentralisation, 
Evaluation des performances, Choix de Tusager, Management par objectif. 

Copyright OECD, 2007. 

Application for permission to reproduce or translate all, or part of, this material should be made to: 

Head of Publications Service, OECD, 2 rue Andre Pascal, 75775 Paris Cedex 16, Prance. 
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PUBLIC SPENDING EFFICIENCY: INSTITUTIONAL INDICATORS IN PRIMARY AND 

SECONDARY EDUCATION 



Frederic Gonand, Isabelle Joumard and Robert Price ^ 



1. Introduction 

1 . This paper presents composite indicators of the institutional and policy characteristics of primary 
and secondary education systems, collated from the questionnaire responses of 26 OECD member 
countries. It is a companion paper of Sutherland et al. (2007), which presents indicators measuring the 
technical and cost efficiency of public spending in the same education sector, which accounts for 9% of 
total public expenditures in OECD countries on average and 3.8% of their GDP. OECD analysis, 
undertaken largely in the context of OECD Economic Surveys, suggests that well-designed institutional 
and policy settings can result in higher public sector efficiency. The indicators described here are based on 
the institutional characteristics which have been identified as most important in that respect, and are 
designed to be used in subsequent analysis accounting for international differences in efficiency. 

2. The paper is structured as follows. In section 2, the conceptual basis of the composite 
institutional indicators is described in terms of the key properties which a priori may help to determine the 
efficiency of public service provision. Section 3 explains how the indicators are computed. Section 4 
presents the results obtained, discusses the associated rankings of countries according to the various 
dimensions of spending efficiency. 

2. Conceptual basis for the institutional indicators 

3. The conceptual basis for describing and classifying the institutional determinants of public sector 
efficiency has been set out in a recent synthesis paper, summarising the experiences of OECD member 
countries,^ and subsequent work relating to fiscal relations between levels of government.^ Broadly, the 
efficiency-determining characteristics can be grouped under three headings: i) the ability to prioritise and 



1. The authors are members of the Economics Department of the OECD. This paper is a revised version of a 
document prepared for a workshop of the Working Party No. 1 of the OECD Economic Policy Committee 
held in October 2006. The authors are indebted to participants to this meeting, to the Delegates of the 
member countries to the Education Committee, and also to Michael Eeiner, Jprgen Elmeskov, Douglas 
Sutherland, Chantal Nicq, Stephanie Guichard, Anne-Marie Brook, Benedicte Larre, Joaquim Oliveira 
Martins and other colleagues for their useful comments. They are grateful to Veronica Humi, Sandra 
Raymond and Paula Simonin for secretarial assistance. The opinions expressed in this paper are those of 
the authors and are not necessarily shared by the OECD. 

2. Joumard I., P.M. Kongsrud, Y-S. Nam and R. Price (2003). 

3. Sutherland, Price and Joumard (2005) ; Joumard and Kongsrud (2003). 
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allocate resources efficiently {efficiency in resource allocation)-, ii) efficiency in managing spending once 
priorities have been set (efficiency in budget management), and Hi) efficiency in service provision (market 
efficiency). The main characteristics of these headings are defined as follows: 

• Efficiency in resource allocation: In making the budget process responsive to priorities, much 
of the emphasis of recent public sector reform in OECD countries has been on imposing a hard 
budget constraint, via a medium-term budget planning focus, accounting transparency and the 
imposition of fiscal rules. These macro-institutional characteristics will obviously help determine 
the allocation of resources to individual programmes such as education and may well affect 
cross-country cost-efficiency comparisons. Allowing for the importance of a centrally imposed 
budget constraint, at the programme level the two most important indicators of an ability to 
allocate resources efficiently have been defined as follows:"^ 

- The degree of decentralisation of responsibilities between central government and 
sub-national public authorities is taken as improving efficiency in the allocation of public 
spending resources insofar as educational needs may differ from one geographical area to 
another and resources should be matched to them. However, decentralisation may become 
counterproductive and reduce efficiency if it is poorly designed, resulting, for instance, in 
overlapping responsibilities between levels of government. 

- Matching resources to specific needs — which may encompass mechanisms to support the 
disadvantaged — may have a favourable impact on overall educational efficiency, notably by 
avoiding “cream-skimming” effects at the aggregate level. Such mechanisms may be required 
in order to make up for the tendency of education systems to under-provide services for less 
able pupils. 

• Efficiency in budget management: Under the heading of efficiency in budget management, two 
crucial efficiency enhancing characteristics may be identified: 

- The extent to which policy is outcome focused allows clear objectives to be set for public 
institutions involved in education, especially if backed by associated evaluation, reward 
and/or sanction systems. 

- The degree of managerial autonomy, especially at the school level, based on flexibility of job 
status, wage setting and budget allocation and disciplined by liberalised outsourcing, where 
possible may also make for greater efficiency in the use of resources. 

• Market efficiency: Productive efficiency is presumed to be related to the degree of competitive 
pressure in service provision, which involves the presence of market signals: 

- Benchmarking may improve service provision by identifying best practices and inefficiencies. 

- Allowing for user choice among alternative providers of educational services may be one of 
the most effective means of giving market signals a role in enhancing the effectiveness of 
public spending in education. This may strengthen competitive pressures and results in 
services which respond better to citizens’ needs — provided that spending follows the user. 



4. These two intermediate indicators of efficiency in resource allocation do not directly measure the strength 

of the budget constraint, but a higher degree of decentralisation is assumed to translate into a higher degree 
of financial responsibility for local decision makers, provided that decentralisation is accurately designed 
and avoids, for instance, overlaps of responsibilities. 
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4. Figure 1 displays the framework of efficiency categories, intermediate indicators and low-level 
indicators which arises from the conceptual breakdown. The method hy which intermediate and composite 
indicators are compiled from this schematic structure is further elaborated in the next section. 

[Figure 1. Main institutional and policy factors potentially affecting public spending efficiency 

in the education sector] 

5. The indicators describe the institutional framework in the public education sector as resulting 
from current regulations. Their construction assumes that the institutions described in the answers to the 
questionnaire are implemented and working in practice according to regulatory prescriptions. While this 
assumption may be optimistic in some cases, relaxing it would bring about even more serious 
methodological problems, related with the measure of the degree of implementation of recent reforms. In 
this context, the interpretation of the indicators should remain cautious. For instance, the favorable bias in 
the indicators might be significant for Portugal. The recent OFCD Country survey for Portugal indeed 
highlighted that the full implementation of recent reforms (increasing the autonomy of the schools or 
favoring benchmarking) remains an important challenge in this country (Guichard and Farre, 2006). In all 
countries, the indicators have to be interpreted as describing the education system when the 
implementation of current reforms is completed. 

6. The institutional indicators are computed at the national level and do not account for possible 
heterogeneity among sub-national authorities. This point applies especially to federal countries and brings 
about some specific issues discussed in Box 1. 



Box 1. Federal countries and institutional indicators computed at the national level 

The institutional indicators presented in this document are computed at the national level. Accordingly, they do 
not account for heterogeneity among sub-national authorities. 

In the case of federal countries, the indicators implicitly average features among different sub-national authorities, 
thus delivering a potentially distorted picture of the institutional framework in the public education sector. For instance, 
if one State/province scores low in “outcome-focused policy” and high in “managerial autonomy” while another 
State/province is efficient for the first item but inefficient for the second, then both States/provinces are relatively 
inefficient in budget management because “outcome-focused policy” and “managerial autonomy” are complementary 
features of the education sector (see below). However, the value of an aggregate institutional indicator computed at 
the national level will be average, overestimating the efficiency of the public education sector of the federal country. 

In the specific case of Belgium, institutional indicators have been computed at the sub-national level of the three 
linguistic communities (Flemish, French speaking and German speaking). Educational systems are almost completely 
different between these communities and the degree of heterogeneity of educational institutions between them can 
reasonably be considered as far higher than in any other OECD country. Indicators computed for Belgium at the 
national level would have been almost meaningless. Moreover, institutional indicators for the Belgian linguistic 
communities can be compared with efficiency indicators computed at the same level, because all data needed to 
compute performance indicators are available at the community level. Since the ultimate purpose of this exercise is to 
analyse the correlation of institutional indicators with efficiency measures, the availability of PISA data at the 
community level in Belgium justified building institutional indicators at the same level. 

For all the other OECD countries, this paper presents institutional indicators computed at the national level, 
mainly because PISA data used to compute perfermance indexes are often not available at the sub-natienal levels. In 
this context, institutional indicators computed at sub-national levels would not have been compared to some efficiency 
indexes. 

The problem asseciated with computing institutional indicators at the national level for federal countries should 
not be overestimated. Heterogeneity among different provinces of a federal country - not taken into account here - is 
prebably much lower than heterogeneity between OECD countries - which is reflected in the indicators of this paper -, 
the more se as the number of sub-national authorities in a given country is high. 
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3. Compilation and design of the institutional indicators 
3.1 Computing the six intermediate indicators for each country 

7. Questionnaire responses have been used to compute six intermediate indicators corresponding to 
the six institutional properties of the public education sector described in section 2. For each country, the 
indicators are computed as weighted means of some of the 21 “low-level” indicators, scaled from 0 to 10, 
with a higher score associated with a more desirable outcome.^ Full details of the coding are given in 
Annex 2 (Tables A.l and A.2). 

8. Constructing an intermediate index requires an assessment of the relative importance ascribed to 
the different low-level indicators, which may vary both across countries, given different institutional 
settings, and over time. To help overcome the difficulties in assigning relative importance to individual 
aspects of institutional and policy efficiency in widely different budgetary frameworks, the approach 
adopted here is based on random weights, as applied in previous OFCD studies constructing indicators of 
financial regulation, product market regulation and sub-central fiscal rules.® These do not assume any 
detailed knowledge of how the various institutional attributes interact, but they allow the identification of 
ranges of possible values for the intermediate index, depending on the different weightings assigned to the 
low-level indicators. 

9. The first intermediate indicator with respect to efficiency in resource allocation deals with 
decentralisation, i.e., the definition and sharing of responsibilities between the central government and 
local authorities (regional as well as local) in the public education sector. An excessively centralised 
framework would undoubtedly leave some local educational needs unanswered. On the other hand, 
decentralisation might also result in undue institutional complexity in some cases. The intermediate 
indicator of decentralisation is thus composed of four low-level indexes, which respectively assess the 
degree of localised decision-making; whether responsibilities are clearly defined between central and sub- 
central authorities; whether they are clearly defined among sub-central authorities, and whether 
responsibilities are allocated consistently between levels of government, avoiding jurisdictional overlaps. 

10. The second intermediate indicator with respect to efficiency in resource allocation - “matching 
resources to specific needs” - measures the ability of the public education system to meet the demand for 
specific educational services and, especially, to avoid the under-provision of educational services to 
students with less favoured socio-economic background. This index is composed of three low-level 



5. The optimal score for all the low-level indicators computed in this paper is 10. In the case of LLI-1 
“Degree of localized decision-making”, a theoretical case could nevertheless be made reasonably in favor 
of an optimal level below 10 since a completely decentralized institutional framework may not be optimal 
in the education sector. Empirically, however, this low-level indicator LLI-1 on decentralisation is 
computed using answers in the questionnaire about 17 educational functions (see Annex 3), all of which 
can (and should) be monitored at a sub-national level. Accordingly, a score of 10 is optimal even for LLI-1. 

6. Starting with low-level indicators, this technique uses 1000 sets of randomly generated weights to calculate 
1000 intermediate indicators. The random weights are drawn from a uniform distribution between zero and 
one and then normalised so as to sum to one. This is equivalent to assuming complete uncertainty about the 
most appropriate value of each of the individual weights used to construct the intermediate indicator. 
Accordingly, the resulting distribution of intermediate indicators reflects the possible range of values given 
no a priori information on the most appropriate value for each of the weights. Confidence intervals and the 
probability of a given country achieving a given rank are calculated from these distributions. The 
confidence intervals are centered on the mean value of each country’s 1000 intermediate indicator values. 
Given that the weights are drawn from a uniform distribution between zero and one, the mean indicator 
values are asymptotically equivalent to indicators calculated using equal weights on each of the low-level 
indicators. 
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indicators. The first two are related to the ability of the educational system to take account of specific 
educational needs when financing schools and rewarding teachers. The remuneration issue may he all the 
more relevant where user choice increases the risk of social segmentation and “cream-skimming” (see 
helow). A third factor widely recognised as enhancing the efficiency of public spending in education 
relates to the age at which the education system becomes selective, which should not be too low.^ 

1 1 . The first intermediate indicator with respect to efficiency in budget management - the 
“outcome-focused policy” index - assesses the extent to which public education systems are based on 
results-focused management rather than on a narrower framework which emphasises conformity to legal 
and procedural rules. The index is composed of three low-level indicators applied to the public school 
sector, considering whether targets are clear, the associated reward and sanction systems are credible, and 
the coverage of the performance assessment mechanisms is sufficient. 

12. The second intermediate indicator under the budget efficiency heading - “managerial autonomy 
at the school level” - assesses the degree of managerial autonomy in the public education sector, especially 
at the school level. A greater autonomy for school managers in their day-to-day operations is a necessary 
condition for a more outcome -focused framework to enhance strongly the efficiency of public spending. 
The index is composed of four low-level indicators, measuring the degrees of flexibility in managing 
human resources, wage-setting and budget allocation and teaching methods and the extent of outsourcing 
possibilities. 

13. The first intermediate indicator of the market efficiency index deals with henchmarking, which 
can help identify both best practices and inefficiencies in schools and thus be an effective tool for 
increasing competitive pressures.* The intermediate indicator is composed of three low-level indicators, 
assessing the geographical and school coverage of benchmarking and measuring its quality. 

14. The second intermediate indicator relating to market efficiency deals with user choice, i.e., the 
possibility for pupils and/or their families to choose the school they prefer. Allowing families a choice 
among alternative schools can strengthen competitive pressures, trigger innovation and result in services 
which respond better to educational needs.^ The user choice indicator is composed of four low-level 
indicators. The first two assess the possibilities of choosing among public schools and between public 
schools and private schools receiving public financial support. A third low-level indicator measures the 
extent to which user choice is limited by some general regulations concerning, for instance, the size of 
school buildings or the availability of public transport. A fourth low-level indicator deals with school 
admission policies and assesses their potential detrimental impact on effective user choice at the school 
level. 



7. Though challenged in some recent studies, empirical research tends to support the view that tracking 
programmes might harm disadvantaged children. 

8. While the positive effect of benchmarking on school efficiency is, in principle at least, hardly doubtful, it 
might not appear immediately in the data. If poorly performing school systems resort more frequently to 
benchmarking, some endogeneity between efficiency and institutional indicators might emerge and, if so, 
would need to be controlled for. 

9. The possible under-provision of educational services to low-skilled pupils (“cream-skimming”) can stem 
either from an inefficient resource allocation framework at the aggregate level or from specific admission 
policies at the school level which lessen the market efficiency of user choice. The intermediate indicator 
“matching resources to specific needs” deals with the first case (see above), whereas the “user choice” 
intermediate indicator is concerned with the second. 
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3.2 Computing the composite indicators 

15. The composite indicators combine, for each country, the six intermediate indicators of spending 
efficiency in the public education sector into a single, aggregate measure. Sub-aggregate measures 
representing the three different types of efficiency (efficiency in resource allocation, efficiency in budget 
management and market efficiency), as described above, can be derived in the same way. By definition, 
the three efficiency types are all encapsulated in the composite indicator. As with intermediate indexes, 
random weights have been applied when computing composite indicators in order to identify the ranges of 
their possible values. 

1 6. Computing the composite indicator and the scores for the three categories of efficiency (as shown 
in Annex 2, Table A.3) by simply averaging the values of the intermediate indexes would deliver a 
distorted picture of the educational system, by downplaying interactions among the institutional properties 
of the intermediate indicators. Accordingly, four types of complementarities between the intermediate 
indexes have been defined and taken into account in the composite indicators: 

- A country ranking high in outcome-focused policy is considered as relatively less efficient if 
it ranks low in managerial autonomy at the school level and/or benchmarking. Setting targets 
is of a limited use as long as local managers are not responsible for meeting them and/or not 
informed of the performance of other schools. Accordingly, a strongly outcome -focused 
policy enhances spending efficiency to the extent that autonomy at the school level and/or 
benchmarking are well-developed. 

- The favourable impact of managerial autonomy on efficiency is conditional on how much the 
education policy is outcome -focused. 

- The effect on market efficiency of wider user choice among schools depends to some extent 
on the quality of benchmarking. 

- The favourable impact of decentralising responsibilities on spending efficiency is also 
partially dependent on whether benchmarking on a national basis is available. 

17. Two methods of computing composite indicators and scores for each category of efficiency have 
been developed which take account of such complementarities (see Annex 2 for more details). Both 
transform the values of the intermediate indexes so as to take account of complementarities and then 
average the transformed values to yield aggregate scores. In the “multiplicative aggregation” method, the 
transformed value of each intermediate index is computed as a geometric mean of the values of the 
indicators defined as being complementary. Though relatively intuitive, this method postulates an arbitrary 
degree of intensity of complementarities between intermediate indicators. In the second “exponential” 
aggregation method, the degree of complementarity can vary and the results can be assessed for their 
sensitivity to the amount of complementarity assumed. 

4. Results 

4.1 Rankings of countries according to six dimensions of spending efficiency in public schools 

18. Figure 2 shows the results obtained for the first component of spending efficiency in resource 
allocation, namely, the intermediate indicator for decentralisation. The left-hand side of Figure 2 shows the 
mean values of each intermediate indicator for each country and the range that contains 90% of the 1000 
random weighted indices. The confidence intervals shown vertically can be interpreted as proxies for the 
degree of consistency, or dispersion, between the low-level indicators used to compute each intermediate 
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index. The right-hand side of Figure 2 matches this measure of dispersion (along the vertical axis) to the 
mean value of the intermediate indicator (horizontal axis). For instance, a consistently decentralised system 
would in principle have a high value of the indicator and a low dispersion of the suh-indices. 

19. As concerns decentralisation, the results depicted in Figure 2 suggest that there is strong and 
consistently designed decentralisation of responsibilities in primary and lower secondary education in 
Denmark and Finland. Centralisation in the public education sector is strong and/or decentralisation is less 
consistently designed in Slovakia, the Czech Republic, Portugal, Austria, Fuxembourg, the Netherlands 
and France. 



[Figure 2. Efficiency in resource allocation: decentralisation] 

20. As seen in Figure 3, the ability of the educational system to match resources to specific 
educational needs, at the aggregate level, appears rather poor in Turkey, Austria, Fuxembourg, Germany, 
Slovakia and Greece, whereas Portugal, France, New Zealand, Japan and Italy lie at the opposite side of 
the spectrum. The figure on the right hand side reveals — particularly for Denmark, Finland, Greece, 
Iceland, Norway, Spain and the United Kingdom — quite pronounced dispersion of sub-indices, suggesting 
that policies are generally not very consistent in matching resources to specific needs. 

[Figure 3. Efficiency in resource allocation: matching resources to specific needs] 

21. Figure 4 shows the results obtained for the first component of efficiency in budget management, 
i.e., the intermediate indicator related to outcome -focused policy. Substantial differences appear among 
OFCD countries in this area. Some member countries define very few objectives (Fuxembourg, Denmark, 
Japan, Norway, Spain, Hungary, Switzerland) and even no objective at all in some cases (Italy, Iceland), 
while regulations in other countries favour strong outcome -focused management practices in the education 
sector (Slovakia, Mexico, Turkey, United States, United Kingdom, Sweden, the Netherlands). Where the 
use of outcome -focused policies is either strong or weak, policies tend to be relatively consistent. In 
intermediate cases, particularly in Belgium (the Flemish community), the Netherlands and New Zealand, 
the relatively high dispersion of sub-indices indicates that policies in this area may be less consistently 
designed. However, the results should be interpreted with some caution since the indicators make the 
implicit assumption that institutional practices are implemented as described, which may not always be the 
case (see section 2). 

[Figure 4. Efficiency in budget management: outcome-focused policy] 

22. A closer look at the low-level indicators (FFI) used to compute the “outcome -focused policy” 
index delivers an interesting stylized fact: where clear outcome targets (FFI-8) exist, consistent reward 
and/or sanction systems associated with educational results (FFI-9) are also often implemented. Figure 5 
shows that both LLI seem to be positively correlated on average among OFCD countries. Though this 
correlation needs to be interpreted with care since the value of one FFI may depend on many others 
factors, it appears to be reasonably significant. This stylised fact suggests that the implementation of 
outcome -focused policy in the educational sector has, on average, been relatively consistent: defining 
targets without setting a reward and/or sanction framework, or the other way round, would not strongly 
foster efficiency in public spending. 



10. To compute an intermediate indicator, low-level indexes are aggregated using a random weight technique 

(see above). Accordingly, a higher dispersion of low-level indexes results in a higher confidence interval 
for the intermediate indicator, which may be interpreted as a measure of the internal consistency of the 
institutions associated with an intermediate index. 
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[Figure 5. Clarity of outcome targets vs credibility of reward/sanctions systems] 

23. As concerns the second component of efficiency of budget management - i.e., managerial 
autonomy at the school level Figure 6 shows that all member countries allow public school managers 
some degree of autonomy. Policies in this area also tend to be reasonably consistent with little marked 
differences among countries (right hand panel). Countries such as the United Kingdom, Hungary, Slovakia, 
New Zealand, Norway, the Czech Republic, Australia, Canada and the United States rank relatively highly 
in this category, whereas the educational systems in Mexico, Luxembourg, Switzerland, Iceland, Greece, 
Germany, Turkey and France do not allow much room for autonomy at the school level. Interestingly, the 
amount of managerial autonomy allowed in Mexican schools is low and — to a lesser extent — this also 
applies in Turkey, leaving some room for improving the consistency of their otherwise strongly outcome- 
oriented educational systems. Likewise, there is some inconsistency in the budget management framework 
in Hungary and, to a lesser extent, the Czech Republic, because managerial autonomy is high while 
education policy remains poorly outcome -focused in both countries. 

[Figure 6. Efficiency in budget management: managerial autonomy at the school level] 

24. Looking at the LLIs used to compute the “managerial autonomy at the school level” index gives 
some more insight about the conditions for a policy fostering autonomy of school managers to be 
consistent. As shown in Figure 7, flexibility in the employment status of teachers in public schools (LLI- 
11), flexibility in setting their remuneration (LLI-12) and the degree to which schools can allocate their 
own budget and choose their teaching methods (LLI-13) are positively and significantly correlated among 
OECD countries. To some extent, this suggests that these three institutional characteristics are 
complementary. 

[Figure 7. Correlation between three dimensions of managerial autonomy at the school level] 

25. Figure 8 shows the results obtained for the first component of market efficiency in educational 
institutions, i.e., benchmarking. A group of countries (Belgium, Switzerland, Austria, Finland, the Czech 
Republic, Mexico, Japan, Spain, Germany) is clearly less advanced in the implementation of benchmarking 
among schools, while Hungary, the United Kingdom, Iceland and the Netherlands appear to have gone 
furthest towards the general use of this practice. 

[Figure 8. Market efficiency: benchmarking] 

26. The questionnaire responses point to a negative relationship between the degree of localised 
decision-making and the quality of benchmarking of pupil performance at the national level (Figure 9).’* 
This result suggests that the decentralisation of primary and lower-secondary education responsibilities to 
sub-national authorities have not been reinforced, on average, by common standards at the national level 
that would have allowed for an efficient benchmarking. 

[Figure 9. Quality of benchmarking of pupils’ performance at the national level vs degree of 

decentralisation] 

27. User choice is another powerful institutional tool fostering market efficiency in primary and 
lower-secondary schools. Results depicted in Figure 10 show that it is very developed and consistently 
applied in Belgium, the Netherlands and Spain - three member countries where private schools represent 
an important proportion of schools. User choice is strongly limited in Turkey, France, Greece, Austria, 



1 1 . The composite indicators take account of the complementarity between the intermediate indicators related 

to decentralisation and benchmarking. 
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Mexico, Switzerland, Japan and Canada. In some countries, such as Italy, Germany and Switzerland, the 
comparatively high variance of suh-indices suggests that user choice policies are less consistently 
designed. 



[Figure 10. Market efficiency: user choice] 

28. Among OECD countries, improved conditions for user choice seem to have developed alongside 
schemes increasing the managerial autonomy at the school level. Eigure 1 1 suggests that where pupils can 
enrol in any public school they wish and where money follows the user in financing public schools, more 
autonomy is also given to public school managers, especially for setting teachers’ wages and deciding 
upon their own budget. 

[Figure 11. User choice among public schools vs schemes increasing the autonomy of school 

managers] 



4.2 Overall assessments 

29. Figure 12 summarises country performance with respect to all six dimensions of spending 
efficiency in the public education sector, comparing the country-specific mean indicators to the OECD 
average and best practice. The scale is chosen so that the outer perimeter of the diagram represents the 
theoretical maximum score. No country can be found where national performance exceeds or equals 
OECD-wide scores for each of the six categories. Some countries exceed or equal the OECD average in 
five out of six institutional categories (United Kingdom, United States, Australia, Denmark, the 
Netherlands, Norway, Sweden). By contrast, Austria does not score better than the OECD average in any 
of the six institutional categories assessed here. 

[Figure 12. Intermediate indicators for each country] 

30. The composite indicators provide, for each country, with a single, aggregate measure of spending 
efficiency in the public education sector. As Figure 13 suggests, the two aggregation methods (see above) 
yield broadly similar rankings for member countries, but differences can be significant in some cases. 

31. Examining the aggregate indicators (Figure 13), the institutional framework emerges as most 
favourable a priori in the United Kingdom, the Netherlands and Sweden, whichever method is used to 
compute the composite indicator. Australia and the United States also rank relatively high compared to the 
OECD average. The institutional framework in primary and lower-secondary education is assessed as less 
favourable in Austria, Switzerland, Luxembourg, Japan, Germany and Mexico. The rankings does not 
appear to be very sensitive to the weighting system (unless, to some extent, for Belgium [the Flemish 
community] and Turkey). As noted above, the confidence intervals of the composite indicator give an 
indication of the overall consistency within national educational frameworks, insofar as a consistent set of 
institutions would have a small confidence interval. Figure 13 suggests that some significant room remains 
for improving institutional consistency in the education sectors of Iceland, Italy, Hungary and Denmark. 

[Figure 13. Composite indicators] 

32. Evidently, the aggregate classification needs to be interpreted in the light of the components. 
Using the conceptual framework where institutional characteristics are grouped under three efficiency 
types, it is possible to assess the performance of a country compared with the OECD average and define 
accordingly where most room remains for improving the educational framework. Figure 14 suggests that 
situations are very heterogeneous in this respect across OECD member countries. However, some features 
emerge and a few tentative country groupings might be defined: 
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- In a first group of countries including the United Kingdom, Australia, Norway, Denmark and 
the Netherlands, the characteristics of the institutional framework seem to he very favourable, 
compared to OECD average, with respect to market efficiency {i.e. benchmarking and user 
choice) and to either resource allocation efficiency or budget management efficiency. Eor 
these countries, performance in any efficiency type is never lower than the OECD average. 

- In another group of countries which includes the United States, New Zealand, Iceland and 
Portugal (with some caveat for the latter, see section 2.), the institutional setting in primary 
and lower-secondary education remains globally favourable to spending efficiency, albeit 
national performance can be lower than the OECD average in one specific efficiency type, or 
higher in only one area out of the three considered in this paper. 

- The overall picture is mixed for Erance, Slovakia, Italy, Canada, Belgium (Elemish 
community), Einland, Spain and Mexico, where the institutional characteristics of the public 
education sector either do not stand out as more (or less) favourable in any of the three 
dimensions of spending efficiency, or where outperformance in one category is offset by a 
lower score in another one. Germany, which was identified as relatively unfavourably placed 
by the composite indicator, belongs to this group. 

- Results are less favourable for the Czech Republic, Greece, Luxembourg, Japan, Turkey, 
Hungary, Belgium (Erench speaking community), Switzerland and Austria. Among these 
countries, Austria, Switzerland, Japan, Luxembourg and the Czech Republic also ranked 
relatively low with respect to the composite index. This group includes several countries 
which are low in market efficiency (Austria, Switzerland, Japan and the Czech Republic), 
while efficiency of resource allocation is a potential problem in Austria, Greece, 
Luxembourg, Hungary and Turkey. The institutional framework is assessed as being 
relatively unfavourable in respect of both market efficiency and budget management in 
Switzerland, and unfavourable in respect of all efficiency types in Austria. 

[Figure 14. Properties of the educational system as regards spending efficiency (relative to the 

OECD average)] 
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ANNEX 1. FIGURES 



1. Main institutional and policy factors potentially affecting public spending efficency in the 
education sector 

2. Efficiency in resource allocation: decentralisation 

3. Efficiency in resource allocation: matching resources to specific needs 

4. Efficiency in budget management: outcome -focused policy 

5. Clarity of outcome targets credibility of reward/sanctions systems 

6. Efficiency in budget management: managerial autonomy at the school level 

7. Correlation between three dimensions of managerial autonomy at the school level 

8. Market efficiency: benchmarking 

9. Quality of benchmarking of pupils’ performance at the national level degree of 
decentralisation 

10. Market efficiency: user choice 

11. User choice among public schools vs schemes increasing the autonomy of school managers 

12. Intermediate indicators for each country 

13. Composite indicators 

14. Properties of the educational system as regards spending efficiency (relative to the OECD 
average) 
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Figure 1 . Main institutional and policy factors potentially affecting public spending efficiency in the education 

sector 
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Figure 2. Efficiency in resource aliocation: decentraiisation 




Figure 3. Efficiency in resource aliocation: matching resources to specific needs 




A/ofe.'The figure gives the average and the range that contains 90% of the 1000 random weighted indices. 
Source: OECD Secretariat calculations based on questionnaire responses. 
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Figure 4. Efficiency in budget management: outcome-focused policy 




A/ofe;The figure gives the average and the range that contains 90% of the 1000 random weighted indices. 
Source: OECD Secretariat calcuiations based on questionnaire responses. 



Figure 5. Clarity of outcome targets vs credibility of reward/sanctions systems 




Existence and clarity of outcome targets 



18 




Flexibility in setting teachers' wage 



ECO/WKP(2007)3 



Figure 6. Efficiency in budget management: managerial autonomy at the school level 




A/ofe;The figure gives the average and the range that contains 90% of the 1000 random weighted indices. 
Source: OECD Secretariat caicuiations based on questionnaire responses. 



Figure 7. Correlation between three dimensions of managerial autonomy at the school level 




Flexibility in employment status of teachers in public schools Flexibility in employment status of teachers in public schools 
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Figure 8. Market efficiency: benchmarking 




A/ofe.'The figure gives the average and the range that contains 90% of the 1000 random weighted indices. 
Source: OECD Secretariat calculations based on questionnaire responses. 



Figure 9. Quality of benchmarking of pupils’ performance at the national level vs degree of decentralisation 



10 n ‘WLD *-HyN- 




Degree of localised decision-making 



20 




Flexibility in employment status of teachers 
in public schools 



ECO/WKP(2007)3 



Figure 10. Market efficiency: user choice 




A/ofe;The figure gives the average and the range that contains 90% of the 1000 random weighted indices. 
Source: OECD Secretariat calculations based on questionnaire responses. 



Figure 1 1 . User choice among pubiic schools vs schemes increasing the autonomy of schooi managers 
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Figure 12 . Intermediate indicators for each country 
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Note: the clear line stands for the OECD average. The crosses stand for OECD best practice for each intermediate indicator. 

The values of the intermediate indicators presented here are not transformed to take account of complementarities (see Annex 2). 




Figure 12 . Intermediate indicators for each country (cont.) 




Note: the clear line stands for the OECD average. The crosses stand for OECD best practice for each intermediate indicator. 

The values of the intermediate indicators presented here are not transformed to take account of complementarities (see Annex 2). 




Figure 12 . Intermediate indicators for each country (cont.) 




Note: the clear line stands for the OECD average. The crosses stand for OECD best practice for each intermediate indicator. 

The values of the intermediate indicators presented here are not transformed to take account of complementarities (see Annex 2). 




Figure 12 . Intermediate indicators for each country (cont.) 




Note: the clear line stands for the OECD average. The crosses stand for OECD best practice for each intermediate indicator. 

The values of the intermediate indicators presented here are not transformed to take account of complementarities (see Annex 2). 





Figure 12 . Intermediate indicators for each country (cont.) 




Note: the clear line stands for the OECD average. The crosses stand for OECD best practice for each intermediate indicator. 

The values of the intermediate indicators presented here are not transformed to take account of complementarities (see Annex 2). 
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Figure 13. Composite indicators 



Composite institutional indicator of public spending efficiency Composite institutional indicator of public spending efficiency 

(multiplicative aggregation) (multiplicative aggregation) 




Composite institutional indicator of public spending efficiency 

(exponential aggregation with low degree of complementarity (gamma=0.2)) 





Composite institutional indicator of public spending efficiency Composite institutional indicator of public spending efficiency 

(exponential aggregation with high degree of complementarity (gamma=0.8)) (exponential aggregation with high degree of complementarity (gamma=0.8)) 
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Figure 14 . Properties of the educational system as regards spending efficiency (relative to the OECD average) 
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Note: For each country, a "high efficiency" score for one category of efficiency means that the two intermediate indicators associated with this efficiency type are 
both higher in this country than the OECD average, and a "low efficiency" score that the two intermediate indicators are both lower than the OECD average. 
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ANNEX 2. CODING, DETAILED RESULTS AND METHODOLOGICAL ISSUES 



A. Coding and Results 



Table A1 . Coding for composite indicators 

Institutional frameworks enhancing efficiency in public pre-primary, primary and lower-secondary education 



Coding for efficiency in resources allocation 



Low-level indicator 



Weight Coding 



Decentralisation 



LU-1 Degree of localised decision-making 

(where D(i) stands for the number of education functions which regional and/or local Institutions are involved in, and 
DD(i) for the number of education functions which central and regional and/or local institutions are involved in) 



LLI-2 Clearly defined responsibilities between central government and sub-national authorities (i.e. no overlapping) 

(where E(i) stands for the number of education functions 
which both regional or local Institutions and central government are involved In) 

LLI-3 Clearly defined responsibilities among and sub-national authorities (i.e. no overlapping) 

(vdiere F(i) stands for the number of education functions 
which both regionai and local institutions are involved in) 

LLI-4 Consistency in the sharing of responsibilities between levels of government 

The institution mainly in charge of determining teachers' starting saiaries and saiary increases is the same as the 
institution mainly in charge of establishing the school overall budget enveloppe and deciding on budget allocation 
within the school 

The institution mainly in charge of opening or closing schools is the same as the institution mainly in charge of paying 
teachers and/or supporting capital expenditures 

The institution in charge of financing newschoois is the same as the institution in charge of financing the maintenance 
of existing schools 

Matching resources to specific needs 

LLI-5 Taking account of specific educational contexts in financing schools 

Some financial adjustements are possible to take account for pupils' special needs - public schools as well as privately 
managed mainly publicly financed schools 

Some financial adjustements are possible to take account for pupils' special needs - public schools only 
No adjustement possible 

LLI-6 Taking account of specific educational contexts in managing teachers 

For each positive response in question 12ter +2.5 

LLI-7 Age of first selection 

(where F stands for the first age of selection in the education system if F<=15, otherwise =10) 10 - 2 * (15 - F) 



10 

5 

0 



10* D(i)/DD(i) 

10-10* E(i)/D(i) 
10-10* F(i)/D(i) 

+3.33 

+3.33 

+3.33 
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Table A1. Coding for efficiency in budget management (cont'd) 



Low-level Indicator 



Weight Coding 



Outcome-focused policy 



LLI-8 Clarity of outcome targets 

Existence of results-focused targets 1 /3 

At least one result-focused target and at least one reward/sanction system 1 0 

No result-focused target, or targets but no reward/sanction 0 

Correspondance 1 target - 1 institution 
Each target applies to only one institution 

There are N(i) institutions in charge of T(i) targets (NB: in this computation, each cross in table 9 counts for an 
institution) 

No resuit-focused target 



1/3 

10 

10*T(i)/N(i) 

0 



Consistency of the institution to which targets are applied 1/3 

Targets mostly apply to the institution(s) in charge of deciding about teaching methods, choosing textbooks, and 
selecting teachers for hire and setting necessary quaiifications 

Targets mostly apply to the institution(s) in charge of deciding about teaching methods, choosing textbooks, or 
selecting teachers for hire and setting necessary quaiifications 
Targets mostly apply to institution(s) not mainly in charge of the 4 items above 
No resuit-focused target 



10 

7.5 

5 

0 



LLI-9 Credibility of reward/sanction systems associated with educational results 



Existence of reward/sanction systems 1/3 

A reward and sanction system exists 1 0 

The existing system features only sanctions or oniy rewards 5 

No sanction and no reward 0 



Consistency of the institution to v\rfiich reward/sanction systems apply 1/3 

No institution with at least one reward/sanction system and no target (i.e. any institution that has at least one ^ ^ 

reward/sanction system also has a target accordingly) 

One institution with at ieast one reward/sanction system but no target 5 

Two or more institutions with at least one reward/sanction system but no target J No sanction and no reward system 0 



Strength of the financial incentives for schools and/or teachers 1/3 

There are financial incentive(s) for schools and/or teachers and. . . 

. . . teachers ' salaries can be adjusted for higher individual performance +5 

. . . school financing takes account of the quality of educational output with. . . 

. . . high performing schools receiving financial reward +2.5 

. . . low performing schools being sanctioned +2.5 

None of the above 0 

LLI-10 Coverage of teachers' and schools' performance assessment 

Public and privately managed but mainly publicly funded schools +5 

Public schools only +2.5 

No assessment/inspection 0 

Teachers and/or schools assessments/inspections are made public +2.5 



(where Z(i) is the fraction of schools in country (i) declaring in the PISA questionnaire (2003) that observation of 
classes by inspectors were used to monitor teaohers) 
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Table A1 . Coding for efficiency in budget management (cont'd) 





Low-level indicator 


Weight 


Coding 




Managerial autonomy at the school level 






LLI-11 


Flexibility in employment status of teachers In public schools 








School involvement in managing human ressources 


1/2 






For each item selected in question 7 a) to e) 




+2 




Flexibility in the management of public schools teachers 


1/2 






(where E is a number of constraints in the employment status of public school personnel) 




10- 2.5* E 


LLI-12 


Flexibility in setting teachers’ wage 








Degree of involvement of schools in wage setting 


1/2 






Schools are not involved in setting teachers' wages 




0 




Schools are involved in setting teachers' wages 
but 2 (or more) others institutions are also involved 




3.33 




Schools are involved in setting teachers' wages 
but 1 other institution is also involved 




6.67 




Only schools are involved in setting teachers' wages 




10 




Degree of flexibility in setting wages 


1/2 






Starting from a score of 5, add the following points if wages of public school teachers . . . 








. . . reflect the number of years of experience 




-2.5 




... reflect academic credential 




-2.5 




. . . reflect individual performance 




+1.67 




. . . vary according to geographic areas 




+1.67 




... vary according to subject areas 




+1.67 


LLI-13 


Involvement of schools In allocating the budget within the schooi and deciding upon teaching methods 








Degree of involvement of schools in budget allocation within the school 


1/2 






Schools are not involved in budget allocation within the school 




0 




Schools are involved in budget allocation within the school 
but 2 (or more) others institutions are also involved 




3.33 




Schools are involved in budget allocation within the school 
but 1 other institution is also involved 




6.67 




Only schools are involved in budget allocation within the school 




10 




Degree of involvement of schools in teaching methods 


1/2 






For each positive response in question 7 k) to n) 




+2.5 


LLI-14 


Outsourcing possibilities 








Coverage of legal restrictions to outsourcing 


1/2 






Legal restrictions apply to auxiliary and core education services 




0 




Legal restrictions apply to core education services only 

(W stands for the compensation of all staff as a % of current PSE spending) 




10*(1-W/100) 




No legal restrictions to outsourcing auxiliary and core education services 




10 




Degree of restriction to outsourcing 


1/2 






Legal restrictions apply to auxiliary and core education services outsourcing 




0 




Legal restrictions do not apply at least to outsourcing auxiliary services 
(X stands for the number of constraints mentioned in question 1 6bis) 




10 -2X 




No legal restrictions to outsourcing auxiliary and core education services 




10 
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Table A1 . Coding for market efficiency (oont'd) 

Low-level indicator Weight Coding 

Benchmarking 

LLI-15 Coverage of benchmarking of pupils' performance 

Geographic coverage 

National 10 

Regional 6.66 

No mandatory benchmarking or others 0 

LLI-16 Quality of national benchnwrking of pupils’ performance 

Coverage by school category 1 /2 

Public and all private schools 10 

Public schools and publicly-funded private schools 6.66 

Public schools only 5 

No mandatory benchmarking or others 0 

Publication of the results 1 /4 

Pupils' results are ajusted and officially published 1 0 

Pupils' results are officially published but not ajusted 5 

Results are not officially published (or no compulsory national benchmarking) 0 

Independance of benchmarking vis-a- vis external influences 1 /4 

Tests are designed and results assessed by independent auditors 1 0 

Others 0 

LLI-1 7 Use of benchmarking at the school level 

Assessments of 15-year-old students are used to compare the school to district or national performance +5 * X(i) 

(with X(i) referring to the fraction of schools of country (I) declaring so in the PISA school questionnaire (2003)) 

Assessments of 15-year-old students are used to compare the school with other schools +5 * Y(i) 

(with Y(i) referring to the fraction of schools of country (I) declaring so in the PISA school questionnaire (2003)) 
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Table A1 (cont'd) Coding for market efficiency 

Low-level indicator Weight Coding 



User choice 

LLI-18 User choice among public schools 

General scope of the user choice among public schools 1/2 

General right to enrol in any public school 1 0 

Derogation needed for enroling in a public school outside a specified geographic area 5 

No choice 0 

Strength of implementation of the "money follows the user" principle among public schools ^^2 
(with C referring to the "capitation basis factor" as in question 1 1bis. The capitation basis 

factor stands for the degree of dependence of public funding to the number of pupils 1 0 * C 

enrolled). 

LLI-19 User choice between public schoois and pubiicly financed private schoois 



Weighted mean of the three items below multiplied by -({2*Z(i)-1)''2)-Hl , where Z(i) refers to 



the percentage of students in public PSE schools 

Scope of the user choice between public schools and publicly financed private schools 1/3 

Private schools can receive some funding from government 1 0 

Private schools can not receive any funding from government 0 

Strength of implementation of the "money follows the user" principle between public and 
privately-managed schools 1/3 

(with C referring to the "capitation basis factor" as in question 1 1bis. The capitation basis 

factor stands for the degree of dependence of public funding to the number of pupils 1 0 * C 

enrolled). 

Admission criteria imposed on publicly financed private schools 1/3 

Private schools can receive funding from government provided that... 

. . . they do not select pupils -h5 

. . . they comply with other admission rules set by national authorities +2.5 

No restriction on admission rules 0 

. . . they can not charge tuition fees -h5 

. . . they comply with other regulation on tuition fees +2.5 

No regulation on tuition fees 0 

Private schools can not receive any funding from government 0 

LLi-20 Liberalised framework for user choice 

For each positive response in question 18bis, starting from 10 -2 

LLi-21 User choice unrestricted by school admission policies 

"Popular" public schools can turn away enrolment demands... 1/2 

... upon their own admission policies 0 

. . . upon other criteria 5 

. . . none of the above (can not turn away enrolment demands) 1 0 

"Popular" publicly financed private schools can turn away enrolment demands... 1/2 

... upon their own admission policies 0 

. . . upon other criteria 5 

. . . none of the above (can not turn away enrolment demands) 10 
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B. Methodological Issues 



33. This section details the methods developed to compute a composite indicator assessing the 
institutional setting in the public education sector as regards its overall ability to bolster spending 
efficiency. Simply averaging the values of the intermediate indexes to yield an overall indicator would 
neglect existing complementarities among the institutional properties of the education system assessed by 
these intermediate indexes. Such complementarities modify the impact on spending efficiency of one 
intermediate index depending on the value of other intermediate, complementary indexes. Two aggregation 
methods taking account of complementarities have been developed in this context. 

Multiplicative aggregation 

34. The composite indicator is computed as a random-weighted mean of transformed values of the 
6 intermediate indexes. The composite indicator is asymptotically equal to:'^ 








‘j 



where I ■ stands for the composite indicator of country i according to the multiplicative aggregation 
method, and I* j is a transformed value of I ■ j , an intermediate indicator for country i as computed from 

low-level indicators using random weights. 7, j is computed as a geometric mean of I ■ j and all its 
complementary indexes: 




1 




J 



where f . is a complementary indicator of 7, j = f ^ j . As mentioned in the main text, the outcome- 

focused policy index has two complementary indexes (managerial autonomy and benchmarking); the 
managerial autonomy indicator has one complementary index (outcome-focused policy), and the user 
choice indicator and the decentralisation indicator each have the benchmarking index as a complementary 
index. 



35. One shortcoming of this method is that the intensity of complementarities cannot be modified. 
This is achieved in the exponential aggregation method described below. 

Exponential aggregation with variable degree of complementarity 

36. As in the previous method, the composite indicator is computed as a random-weighted mean of 
transformed values of the 6 intermediate indexes. It is asymptotically equal to: 



12. Given that the weights are drawn from a uniform distribution between zero and one, the mean indicator values are 

asymptotically equivalent to indicators calculated using equal weights on each of the low-level indicators. 
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ij 
j=l 

where I- stands for the synthetic indicator of country i according to the exponential method of 
aggregation and I ^ j is a modified value of the intermediate indicator I ■ j of country i. 

7, j is computed from the values of all its complementary indexes 7, ^ ^ (with 7, ^ h j\) according to 
the following formula:'^ 

iZ = “7^^^ \ ) + exp(-/( 7 ) min 7,. . , )) 

where ( 0 ; 1 ) is a parameter measuring the intensity of complementarity among intermediate indexes 
and the function f(y) is increasing.'"^ This specification allows for different degrees of 
complementarity ( 7 ) in computing the impact on the value of the composite indicator, depending on the 
dispersion between complementary intermediate indicators. This is explained helow taking a few examples 
(see Figure Al): 

- In Mexico, a high dispersion between complementary intermediate indicators lowers the 
value of the composite indicator. Mexico ranks high on outcome-focused policy (7.0) but low 
on benchmarking (1.8) and managerial autonomy (1.4). As argued in the main text, such a 
country may not actually be efficient. With the above mentioned specification, the 
transformed value of the “outcome-focused policy” intermediate index tends to 1.4 if /—>1 
and lies between 7.0 and 1.4 depending on the intensity of complementarities as measured by 
the value of / . Overall the value of the Mexican composite indicator is lessened, all the more 
as 7 is high. 

Likewise, the composite indicator in Denmark is detrimentally affected by the low level of 
the “outcome -focused policy” index which reduces the value of the transformed “managerial 
autonomy” indicator, and thus the value of the composite indicator. 

- In Sweden and France, the degree of dispersion between complementary indexes is low, 
suggesting a relatively higher institutional consistency in the public education sector. With 
the above mentioned specification, the value of the composite indicator is not strongly 

This specification is a modified version of a function proposed by Blackorby, Bossert and Donaldson 
(2005) who model a welfarist social choice function aggregating individual utilities and taking account of 
the social planner’s aversion to inequality. See Blackorby, Bossert and Donaldson (2005). 

The function f{y) is such that lim/( 7 ) ^ ; lim/( 7 ) ^ +oo and /(0.5) = 0 . In practice, we used 

>0 y—>l 

fir) = -[tan(;T 7 )J and defined 7,. ^^5 as 7,. ,,^ 0.5 = 0-5 * (Iij,r=oA 9 + Iij,y=o. 5 i) ■ 

In this specification, the implicit weights of I ■ and min I ■ in the computation of the transformed 

value of the intermediate index are not the same, thanks to the use of the exponential: 
f j * min7,. ^ ^ exp(- /( 7 ) 7 ,. exp(- /( 7 )mm 7 ,. J. 



13. 



14 . 
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affected by the degree of complementarity ( 7 ) and remains close to the simple average of the 
non-transformed intermediate indexes /, , . 



Figure At . Composite indicator as a function of the degree of complementarity between intermediate indexes 

(exponential aggregation) 
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ANNEX 3 : QUESTIONNAIRE SENT TO THE OECD MEMBER COUNTRIES 



Policy and institutional settings in the education sector 

37. The main objective of the questions included in this sub-section is to build relevant indicators for 
the policy and institutional settings which may help in understanding why different measures of efficiency 
for public service providers vary across countries. 

38. When were the last major reforms implemented in the education sector? What were the key 
features of these reforms? 



The sharing of responsibilities in education across levels of government and other bodies 

39. This set of questions aims at building indicators on the degree and type of decentralisation. 



Who is involved in: 
















Central 

government 


Regional 
government 
(if any) 


Local 

government 


Trade 

unions 


Schools 


Families, 
residents, local 
businesses 


Personnel management 
a) Selecting teachers for hire 


□ 


□ 


□ 


□ 


□ 


□ 


b) Firing teachers 


□ 


□ 


□ 


□ 


□ 


□ 


c) Setting necessary qualifications 


□ 


□ 


□ 


□ 


□ 


□ 


d) Establishing teachers’ starting 


□ 


□ 


□ 


□ 


□ 


□ 


salaries 

e) Determining teachers’ salary 
increases 


□ 


□ 


□ 


□ 


□ 


□ 


Infrastructure development 
f) Financing new schools and other 
buildings 


□ 


□ 


□ 


□ 


□ 


□ 


g) Financing the maintenance of 
existing schools and other buildings 


□ 


□ 


□ 


□ 


□ 


□ 


Financial resources 
h) Establishing the school overall 
budget envelope 


□ 


□ 


□ 


□ 


□ 


□ 


i) Deciding on budget allocation within 
the school 


□ 


□ 


□ 


□ 


□ 


□ 


Curriculum and instruction methods 
j) Setting curricula 


□ 


□ 


□ 


□ 


□ 


□ 


k) Deciding about teaching methods 


□ 


□ 


□ 


□ 


□ 


□ 


1) Setting instruction time 


□ 


□ 


□ 


□ 


□ 


□ 


m) Choosing textbooks to be used 


□ 


□ 


□ 


□ 


□ 


□ 


n) Designing education programmes 


□ 


□ 


□ 


□ 


□ 


□ 


o) Setting catchment areas 
(if any) 


□ 


□ 


□ 


□ 


□ 


□ 


Student policies 

p) Setting rules for student admissions 


□ 


□ 


□ 


□ 


□ 


□ 


q) Opening or closing schools 


□ 


□ 


□ 


□ 


□ 


□ 



40. Please signal whether any significant reform in the allocation of responsibilities across levels of 
governments and agents has been implemented over the past decade. Please also note whether the 
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responses to the questions on personnel management are the same for government-dependent private 
schools with teaching personnel paid hy a government agency. 



Setting targets for educational institutions 

41. The management of public spending programmes has shifted focus from the amount of resources 
used to deliver the service and the results/outcomes achieved. This set of questions aims at measuring the 
degree to which results-focused targets are used, the nature of these and the arrangements in place to 
enforce them. 



The setting of results-focused targets 





Yes 


No 


Have results-focused targets been adopted in your country for primary and lower secondary education 
institutions? 


□ 


□ 


If so, what are these results-focused targets and to which bodies are they applied? 

(tick all boxes that apply) 









School 


Schools 


Local or regional 


Ministry of 


Others 




employees 




governments 


education 


(Including 

agencies) 


Minimum number or percentage of pupils 

achieving a given level of 

proficiency/qualifications 

Same as previous question, taking into 

account the socio-economic background 

of pupils and/or knowledge base when 

they enrol in the school 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Maximum number or share of repeaters 


□ 


□ 


□ 


□ 


□ 


Maximum time of replacement for 
teachers on leave 


□ 


□ 


□ 


□ 


□ 


Truancy rates 


□ 


□ 


□ 


□ 


□ 


Others (please specify) 


□ 


□ 


□ 


□ 


□ 



42. What has been the experience with results-oriented targets for educational institutions? To what 
extent have targets been complied with? What recent reforms have been implemented? 
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Administrative reward and sanction systems 



Yes No 

Are there administrative incentives (sanctions or support) associated with educational results as evaluated 



against the targets? 








□ 

□ 


If your country has any administrative incentives, what are they and to which body are they applied? 






(tick as many as apply) 






















Others 




School 

employees 


Schools 


Local or regional 
government 


Ministry of 
education 


(including 

public 

agencies) 


Can employees in charge of 
implementing the targets be dismissed 
in the case of sub-par performance? 


□ 


□ 


□ 


□ 


□ 


Can employees in charge of 
implementing the targets be promoted in 
the case of high performance? 


□ 


□ 


□ 


□ 


□ 


Has the frequency of inspection 
increased in the case of sub-par 
performance? 


□ 


□ 


□ 


□ 


□ 


Is freedom in decision making restricted 
in the case of sub-par performance? 


□ 


□ 


□ 


□ 


□ 


Are there other forms of incentives 
(if so, please specify)? 


□ 


□ 


□ 


□ 


□ 



Financing principles for educational institutions 



43. This set of questions aims at clarifying what are the main funding principles for public and 
privately-managed schools. 



Can privately-managed schools receive some funding from the general government? 


Yes No 

□ □ 


If the answer Is “yes”, please specify the conditions for funding: 


Tick as many 
boxes as 
apply 


a) Compliance with admission rules set by the national authorities. 


□ 


b) Inability to charge tuition fees. 


□ 


c) Compliance with regulations on tuition fees. 


□ 


d) The curriculum is set by the government 


□ 


e) Inability to select pupils. 


□ 


f) Compliance with other regulations 


□ 


(if so, please specify) 
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Does the financing of schoois refiect, at ieast partiy, the quantity and/or quaiity 
of services deiivered? 

If the answer is “yes”, tick as many as apply: 


Publicly 

managed 


Privately 
managed but 
mainly publicly 
financed 


Yes No 

□ □ 


Yes No 

□ □ 


If schools are financed by public authorities on the basis of the number of pupils, please 
specify the method: 

-- on a pure capitation basis (i.e. public funding is strictly dependent on the number 
of pupils enrolled). 

- largely on a capitation basis (i.e. should one pupil move from a public school to a 
mainly publicly funded school, between 66 and 99% of the cost of educating a 
child in a public school would be reallocated from the origin to the destination 
school). 

-- partly on a capitation basis (i.e. should one pupil move from a public school to a 
mainly publicly funded school, between 33 and 66% of the cost of educating a 
child in a public school would be reallocated from the origin to the destination 
school). 

-- on a “very soft” capitation basis (i.e. should one pupil move from a public school to 
a mainly publicly funded school, between 0 and 33% of the cost of educating a 
child in a public school would be reallocated from the origin to the destination 
school). 

- with some adjustment to account for pupils’ special needs and/or their socio- 
economic backgrounds. 


□ 

□ 

□ 

□ 

□ 


□ 

□ 

□ 

□ 

□ 


If the quaiity of educational outputs is accounted for in financing individual schools, please 






specify the method. 


□ 


□ 


- High performing schools receive a financial reward? 


□ 


□ 


- Low performing schools receive extra financial support to allow them to improve 






the quaiity of services? 


□ 


□ 


- Other output-quality incentives apply? (If so, please specify) 


□ 


□ 


There are other types of key quality and quantity criteria applicable to the financing of 






public school? If so, please specify 


□ 


□ 
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Management of schools 

Employment status, mobility and wage setting in public schools 

44. Implementing a results-oriented management approach has often been accompanied hy changes 
in human resource management. 



Please define the employment status of public school personnel according to the following 
characteristics: 


Yes 


No 


Public school teachers have life-long job contracts. 


□ 


□ 


There are regulatory constraints reducing the geographical mobility of public school 
personnel to reflect changing needs (e.g. demographic developments) 

If so, please specify. 


□ 


□ 


The pension system hinders the mobility of staff between the public and private education 
sectors (e.g. non-portability of pension rights). 


□ 


□ 


Other regulatory constraints hinder the mobility of staff between the public and private 
education sectors. 

If so, please specify. 


□ 


□ 



Please define the wage-setting process of public school personnel along the following lines: 


Yes 


No 


Wages of public school teachers reflect: 






- The number of years of experience 


□ 


□ 


- Academic credentials 


□ 


□ 


- Individual performance (if so, please specify below) 


□ 


□ 


- Others 


□ 


□ 


(please specify) 


□ 


□ 


There are important differences in teachers’ wages across regions or municipalities (for 






similar positions, working time and qualifications). 


□ 


□ 


There are important differences in teachers’ wages across subject areas (for similar 






positions, working time and qualifications). 


□ 


□ 



Are public school teachers working in areas with children from a disadvantaged socio- 
economic background given specific support? 


Yes 


No 


□ 


□ 


If the answer is “yes”, please specify: 


Tick all boxes that apply: 


Wages are adjusted upward 


□ 


□ 


Teachers are given extra training 


□ 


□ 


Working hours are shorter 


□ 


□ 


Class size is generally smaller 


□ 


□ 



45. Please note any comments you may have on the joh status and wage-setting arrangements for 
schools’ personnel and discuss recent reforms in your country in this area. Please also note whether the 
responses to the questions on the management of schools are the same for government-dependent private 
schools with teaching personnel paid hy a government agency. 



Using market-like instruments to improve incentives for educational institutions 
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Benchmarking 

46. This set of questions aims at measuring the extent to which measures of school performance are 
made public and their main features. 



Are pupils regularly submitted to standardised tests? 


Yes 


No 


-- At national level 


□ 


□ 


-- At regional level 


□ 


□ 


If the answer is at a national level “yes”, tick as many as apply: 


tick all boxes 


that apply 


a) Tests are mandatory for all public schools. 


□ 


b) Tests are mandatory for all private schools. 


□ 


c) Tests are mandatory for private schools only if public funds are their dominant financial resource. 


□ 


d) The results of individual schools’ tests are officially published. 


□ 


e) The value a school adds to each pupil’s performance is evaluated and these ’’corrected” results are 
published and promoted at an official level. 


□ 


f) Tests are designed and results assessed by independent auditors. 


□ 


g) Test results are reflected in decisions on public funding decisions for individual schools. 


□ 


And: 

- High performing schools receive more resources or other form of financial reward. 


□ 


- Poor performing schools receive more resources, with the objective to enable them to improve results in the 
future. 


□ 


h) Poor performing schools are sanctioned, either financially or administratively (e.g. dismissal of school 
principal). 


□ 



Is the performance of teachers and/or schools regularly scrutinised through personnel assessment or visits from 
inspectors? 


Yes 

□ 


No 

□ 


If the answer is “yes”, tick all boxes that apply: 


tick all 


boxes that 




apply 


a) Assessments or visits are performed only in publicly managed schools? 


□ 


b) Assessments or visits are performed in both public and privately managed, but mainly publicly funded 
schools. 


□ 


c) Assessments are made public. 


□ 


d) Frequency depends on past performance. 


□ 


e) How frequent are assessments? (please give average number of years between inspections) 


□ 



47. Please note any comments you may have on benchmarking in the education sector and on recent 
reforms in this area. In particular, has benchmarking contributed to an improvement in education outcomes 
in your country? If so, how and why? Are there studies and/or data on this subject? If there is no, or 
limited, benchmarking of schools or if results are not made public, please give the rationale behind this 
policy. 



Outsourcing 

48. This set of questions aims at identifying whether some constraints hinder the development of 
outsourcing in the education sector. 
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Yes 


No 


Are there legal restrictions preventing public schools from sub-contracting: 






• Auxiliary services (e.g. canteen or library services)? 


□ 


□ 


• “Core" education services (e.g. foreign languages courses)? 


□ 


□ 




tick all boxes 


If sub-contracting Is allowed, do any of the following factors act as constraints: 


that apply 


a) The status of employees (e.g. the predominance of permanent contracts) Is an Impediment to outsourcing. 


n 


b) Segments of the education sector are excluded from the tendering rules applying to public procurement. 


n 


c) Public procurement policies are used to protect local and/or small enterprises. 


n 


d) The tax system distorts the choice between In-house provision and outsourcing (e.g. VAT provisions for 




public bodies). 


□ 


e) Other regulatory constraints apply (e.g. public schools are required to buy these services from public Institution 






If so, please specify 


□ 



49. Has outsourcing by educational institutions contributed to improved cost efficiency for public 
spending on education in your country? Give a brief explanation. Are there any studies and data available 
on this subject for your country? 



Family choice for schools and competition between schools 

50. This set of questions aims at examining the degree of family choice for schools and the extent to 
which schools can respond to demand preferences. 



Do families have a choice among publicly managed schools at the primary and lower 
secondary education levels? 


Yes 

□ 


No 

□ 


If the answer is “yes”, specify the principles for both public schools and 
privately managed but mainly publicly funded schools: 


Public 

schools 


Privately managed and 
mainly publicly funded 
schools 


a) Pupils are given a general right to enrol in any school they wish. 


□ 


□ 


b) Pupils have to apply for a derogation to enrol outside his/her zone (rather than being 






given a general right to enrol in any school they wish). 


□ 


□ 


c) Others 


□ 


□ 


(please specify) 
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Are there serious constraints impeding “popular” public schools and/or publicly financed private schools from 
meeting demand? 


Kes 

□ 


No 

□ 




tick as many as 


If the answer is “yes”, please specify the nature of the constraint: 


apply 


a) Financing constraints (e.g. because the money does not fully follow the pupils) 


□ 


b) Physical constraints (e.g. norms imposed by the central government on the size of school buildings) 


n 


c) Public transport arrangements for pupils from home to schools discourage families from exercising their 




school choice options 


□ 


d) Regulatory constraints (e.g. the share or number of pupils enrolled in privately managed schools is 






controlled by the government) 


□ 


If so, please specify 






e) Other constraints 






If so, please specify 


□ 



Are “popular” public schools and publicly financed private schools given the possibility of 
turning away the enrolment demand of some pupils? 


Yes 

□ 


No 

□ 


If the answer is “yes”, please specify: 


Tick as many as apply 


a) Schools decide upon their own admission policies and which pupils to admit 

b) Family residence is the predominant criterion for admission 

c) Admissions are determined on a “first come, first served” basis. 

There are other criteria which can be used by schools 

If so, please specify 


Public 

schools 

□ 

□ 

□ 

□ 


Privately managed but 
mainly publicly funded 
schools 

□ 

□ 

□ 

□ 



51. Please add any comments you may have on family choice and competition among schools, as 

well as on recent reforms in this area. In particular, has school choice, if at all, improved education 
outcomes and if so why? Please refer to data and studies on this subject where available. If there is no, or 
limited, school choice, why? 



Final comments on the policy and institutional settings which may influence efficiency 
Note: Replying to this section is optional 

52. Please use the box below to provide any additional comments on the policy and institutional 
settings which may have had an important influence on the efficiency of public spending on education, or 
append your comments on a separate sheet. 



53. Finally, if there are any studies that draw a link between efficiency in public spending on 
education and the policy and institutional settings in your country, we would welcome information on 
these studies. Please provide web-links to abstracts or the reports themselves in the box below. 
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Measuring efficiency of public spending on primary and lower secondary education 

54. Data available from the Education at a Glance and PISA databases will be used for the estimation 
of efficiency. Elowever, we would welcome information on national studies measuring the efficiency in the 
provision of education services. In addition, we would be interested in studies that measure the progress of 
cohorts through education (for example, using information on the number of students repeating a year or 
results from intermediate tests), changes in completion rates, and studies linking educational attainment to 
growth in labour productivity or real earnings. Please provide web-links to abstracts or the reports 
themselves in the box below. 



55. The measurement of spending efficiency may be complicated by tax expenditures and private 
spending on education outside educational institutions (such as for private tutoring). If national estimates 
are available for these factors, please provide these data including some information on the methodology 
(such as the revenue forgone approach for tax expenditures). Please note whether tax expenditures are 
refundable or non-refundable and, if possible, their importance by level of education (pre -primary, primary 
and lower secondary). 
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